We investigate an experiential learning paradigm for acquiring an internal model of intuitive physics. Our model is evaluated on a real-world robotic manipulation task that requires displacing objects to target locations by poking. The robot gathered over 400 hours of experience by executing more than 50K pokes on different objects. We propose a novel approach based on deep neural networks for modeling the dynamics of robot's interactions directly from images, by jointly estimating forward and inverse models of dynamics. The inverse model objective provides supervision to construct informative visual features, which the forward model can then predict and in turn regularize the feature space for the inverse model. The interplay between these two objectives creates useful, accurate models that can then be used for multi-step decision making. This formulation has the additional benefit that it is possible to learn forward models in an abstract feature space and thus alleviate the need of predicting pixels. Our experiments show that this joint modeling approach outperforms alternative methods. We also demonstrate that active data collection using the learned model further improves performance.

We propose information gating as a way to learn parsimonious representations that identify the minimal information required for a task.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

We found that training an inverse model is crucial for learning good representations. On the first row,alevel from each environment that one-shot PPGS fails tosolve(thewhitearrowsrepresent thepolicy). Iterative Model Improvement In general settings, collecting training trajectories by sampling actions uniformly atrandom does not grant sufficient coverage ofthe state space. GLAMORGLAMOR [34] learns inverse dynamics to achieve visual goals in Atari games. The only difference withPPGS in terms of settings is that we allowGLAMORto collect data on-policy and for more interactions (2M).

In this paper we have focused on the backward (or feedback) DNI, but there is another interesting paradigm between two neural networks dubbed "forward" DNI. Here we describe this variant of the model and below its link to the cerebellum. The difference to backward DNI is that now the synthesiser predicts forward activity, not backward. Though more nuanced, the goal of forward DNI as presented in [5] is also to hasten learning. As an example, suppose we have a feedforward network as the main model and equip a backward synthesiser (one which predicts same layer error gradients) at each layer as well as a forward synthesiser which projects from the original network input x onto each layer.

Empirically, S-TEC restructures meaningfully the within-and between-class representations.